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Preferential attachment schemes, where the selection mechanism 
is linear and possibly time-dependent, are considered, and an infinite- 
dimensional large deviation principle for the sample path evolution of 
the empirical degree distribution is found by Dupuis-EUis-type meth- 
ods. Interestingly, the rate function, which can be evaluated, contains 
a term which accounts for the cost of assigning a fraction of the total 
degree to an "infinite" degree component, that is, when an atypical 
"condensation" effect occurs with respect to the degree structure. 

As a consequence of the large deviation results, a sample path a.s. 
law of large numbers for the degree distribution is deduced in terms 
of a coupled system of ODEs from which power law bounds for the 
limiting degree distribution are given. However, by analyzing the rate 
function, one can see that the process can deviate to a variety of atyp- 
ical nonpower law distributions with finite cost, including distribu- 
tions typically associated with sub and superlinear selection models. 

1. Introduction and results. Preferential attachment processes are graph 
networks which evolve in time by linking at each time step a new node to a 
vertex in the existing graph with probability based on a selection function of 
the vertex's connectivity. Such schemes have a long history in various guises 
going back to [50] and [51]; cf. surveys [40, 49]. More recently, Barabasi 
and Albert (BA) in [4] proposed that versions of these processes, where the 
selection function is an increasing function of the connectivity, may serve 
as models for growing real-world networks such as the world wide internet 
web, and types of social structures. 

For instance, in a "friend network," a newcomer may have a predilection 
to link or become friends with an individual with high connectivity, or in 
other words, one who already has many friends. An important property of 
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such reinforcing networks is that when the selection function is in a hnear 
form, asymptotically as time grows, the proportions of nodes with degrees 
1, 2, . . . , fc, . . . converge to a power-law distribution {q{k) :k >1) where < 
liiRk^^ q{k)k^ < oo for some 6 > 0. We will say that a network with such 
a law of large numbers (LLN) property is "scale free." Since it has been 
observed that the sampled empirical degree structure in many real-world 
networks has a "scale-free" form, such preferential attachment processes 
have become quite popular in several ways; see [1, 3, 5, 12, 14, 16, 17, 26, 
30, 40, 43-45] and references therein. 

To illustrate more clearly the possible phenomena, consider the following 
basic example. 

Example 1.1. Initially, at time n = 1, the network Gi is composed of 
two vertices with a single (undirected) edge between them. At time n = 2, 
a new vertex is attached to one of the two vertices in Gi with probability 
proportional to a function of its degree to form the new network G2- This 
scheme continues: more precisely, at time n + 1, a new node is linked to 
vertex x G G„ with probability proportional to w{dx{n)), that is, chance 
w{dxin)) / ^y^Q^ w{dy{n)), where dz{n) is the degree at time n of vertex z, 
and w = w{d) -.fi ^ M_|_ is the selection function. 

In this way, since the initial graph is a tree, all later networks G„ for 
n > 1 are also trees. Let now Zk{n) be the number of vertices in G„ with 
k links, Zk{n) = J2yeGn ^('^»/('^) ~ '^^^ describe a trichotomy of 

growth behaviors corresponding to the strength and type of the selection 
function w [36]. 

First, when w is linear, say w{d) = d + a for a > — 1, the system is "scale- 
free." As is well understood in the literature (cf. [30], Chapter 4), the mean 
values (Mfc(n) = E[Zk{n)] :k>\) satisfy rate equations in the time index 
n>l which can be solved to show lim„^oo Mii.{n)/n = q{k) for k>l where 
q is in power-law form with = 3 + a. 

Later, in [10], when a = 0, a concentration inequality was used to show 
convergence in probability, lim„-|-oo (n)/n = q{k) for fc > 1. We will call 
the a = model the "classical BA process" as it was the model originally 
analyzed in [4]. Also, for all a > —1, Polya urn/martingale ideas, and em- 
beddings into branching processes have given alternative proofs which yield 
a.s. convergence; see [2, 41, 48]. 

However, in the sublinear case, when w{d) = d''' for < r < 1, although it 
was shown that a.s. lim^^oo Zk{n)/n = q{k), this LLN limit q is not a power 
law, but in stretched exponential form [36, 48]: for k>l, 
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Asymptotically, logq{k) ~ — (/i/(l — r))k^~''' as k']'oo. On the other hand, 
when r = 0, the case of uniform attachment when an old vertex is selected 
uniformly, an a.s. LLN can also be similarly obtained where q is geometric. 
q{k) = 2-'' for A:>1. 

In the superlinear case, when w{d) = cT for r > 1, "explosion" or a sort of 
"condensation effect" ' happens in that in the limiting graph a random single 
vertex dominates in accumulating connections. In particular, the limiting 
graph is a tree where there is a single random vertex with an infinite number 
of children; all other vertices have bounded degree, and of these only a finite 
number have degree strictly larger than r/(r — 1); cf., for a more precise 
description, [36, 46]. Moreover, a LLN limit, lininfoc EZ^in) /n = q{k), is 
argued where q is degenerate in that = 1 but q{k) = for A; > 2; cf. [36] 
and [30], Chapter 4. Such a limit implies, in the superlinear selection process, 
that most of the nodes at step n are leaves. 

Since the work of Barabasi and Albert [4], much effort has been devoted to 
understand the degree and other structures in generalized versions of these 
graphs. A partial selection of this large literature includes: more on degree 
structure [23, 24, 31, 32, 34, 37, 38]; growth and location of the maximum 
degree [2, 21, 42]; spectral gap and cover time of a random walk on the 
graph [19, 39]; width and diameter [9, 22, 35]; graph limits [6, 8, 11, 47]. 

Connection between urns and degree structure. If, however, one focuses 
only on the degree structure of the growing network, then it may be helpful 
to view the degree distribution evolution in terms of "balls-in-bins" or "Polya 
urn" models. For instance, in the previous example, every new connection 
that a vertex gains can be represented by a new ball added to a corresponding 
urn in a collection of urns. More precisely, at time n = 1, there are two urns, 
each possessing one ball, in the initial collection U{0). At time j ' + 1, a new 
urn with one ball is included in the collection, and also one ball is added to 
an existing urn x G U{j) with probability proportional to w{bx) where is 
the number of balls in urn x. Then, Zk{n) translates to the number of urns 
in U{n) with k balls for k>l. 

A comprehensive form of such an urn model was formulated by Chung, 
Handjani and Jungreis (CHJ) in [15], motivated by the work in [27] and [36] 
on the organization of web tree-graph models. See also [7] and [42] for other 
work connecting urns to degree structure. 

The CHJ model is as follows. Given an initial finite collection of urns each 
containing one ball, at subsequent times, with probability p, a new urn with 
one ball is created and added to the collection or, with probability 1 — p, 
a new ball is put in one of the existing urns x with probability proportional 
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to (bxY where bx is the number of bahs in x. It was proved in [15], among 
other results, when r = 1 and p> 0, analogous to linear selection preferential 
attachment graphs, that the empirical urn size distribution converges to a 
power law with 6 = 1 + {1 — p)~^. 

In this context, our purpose is to study a generalized preferential attach- 
ment process of urns, where at each time step a new urn is created and a new 
ball is added to it or an existing urn according to a time dependent linear 
selection function, which includes the evolving degree structure of linear se- 
lection preferential attachment model discussed above, and also a version of 
the r = 1 CHJ urn model [15]. We defer to Section 1.1 the exact description 
of our scheme. 

As mentioned in [28], understanding preferential attachment or urn sys- 
tems, where the selection function depends on time, allows for more realistic 
models given real world networks are time-dependent. However, it appears 
most of the work on time-dependent schemes consists of rate equation for- 
mulations ([25], Section E), [36] and related work, in models where at each 
step a random number of links or balls may be added to the structure [2, 18]. 

Given this background, detailing the large deviation behavior of the de- 
gree distribution in time-dependent preferential attachment schemes is a 
natural problem which gives much understanding of typical and, in partic- 
ular, atypical evolutions. We remark, even in the usual time-homogeneous 
models, large deviations of the degree structure is an open question. 

Previous large deviation work in preferential attachment models has fo- 
cused on one-dimensional objects, for instance, the number of leaves [13], 
or the degree growth of a single vertex with respect to dynamics where any 
vertex may attach to a newly added vertex with a small chance [21]. See 
references cited in [13] for large deviations work with respect to other types 
of random trees and balls-in-bins models. 

Our main work in this article includes an infinite-dimensional sample path 
large deviation principle (LDP) for an array of empirical urn ball size dis- 
tributions {{ZJ^(j) /n:k>0):0<j< n}„>i, when the initial configuration, 
not necessarily fixed, satisfies a limit condition (Theorem 1.4). Here, ZJ^{j) 
stands for the count of urns with k balls at time j in the nth row of the 
array. Part of these results is a finite-dimensional LDP with respect to the 
numbers of urns with less than d balls for d <oo (Theorem 1.2). 

As an application of the large deviations results, we obtain an a.s. sam- 
ple path LLN for the urn counts in terms of a system of coupled ODEs 
(Corollary 1.7), which, for homogeneous schemes complements fixed time 
LLNs mentioned earlier, and gives a different way to derive them aside from 
the rate equation method mentioned in Example 1.1. Finally, the LLN limit 
trajectories are shown to have power law-type behavior in terms of bounds 
(Corollary 1.9), although the general behavior can interpolate between these 
bounds; see Figure 1. 
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Interestingly, the infinite-dimensional rate function I°° can be calculated 
on scaled urn ball size path distributions ^ = : > 0) : < t < 1}. 

Here, since in our model, exactly one ball is added to the urn collection at 
each microscopic time, ^k{t)/{t + c) is the fraction of urns with size k at 
macroscopic time t > where c = X]fc>o'?A:(0) is the initial mass, that is the 
scaled initial number of urns. It is natural then to ask which trajectories ^ 
have finite cost, < oo. 

It turns out "no mass can be lost," that is, all finite cost paths ^ are 
such that the proportions {S,k{i) / {i + c)} k>o form a probability distribution, 
Ylik>o^k{t) / {t + c) = 1. Also, a variety of nonpower law distributions can 
be achieved with finite rate at any time < t < 1, including geometric and 
stretched exponential distributions discussed in Example 1.1. 

Intriguingly, on the other hand, "some of the weight may be lost" in cer- 
tain finite rate trajectories, that is, the scaled mean urn ball size of the sys- 
tem may satisfy a "weight loss" property at a time < t < 1, Ylik>Q ^Ck{t) /(^ + 
c) < 1, where c = ^k>o^^k{^) is the scaled initial total urn ball size, even 
though the pre-limit quantity equals 1 at all steps in the urn growth scheme. 
We dub a trajectory with this "weight loss" property at some time < 
t < 1 as being "condensed." For instance, a "condensed" path arises when 
c = c = and a finite number of the urns take in eventually all the balls. In 
this case, almost all the urns created are empty, and the associated path sat- 
isfies Co(i) = t for < t < 1, = for A; > 1, and k^k{t) = 0. It turns 
out this path, associated with superlinear selection preferential attachment 
models (cf. Example 1.1), has finite cost. 

Moreover, the rate function I°° contains a term which measures the cost 
of "condensation" when some of the flow of urn ball size in the scaling limit 
escapes toward urns with "infinite" size. In addition, we point out, at any 
time < t < 1, LLN distributions arising from either sublinear or super- 
linear selection preferential attachment models may be achieved with finite 
cost. One might interpret that although the linear selection process is typ- 
ically "scale-free," since it is between, in a sense, sublinear and superlinear 
selection models, its atypical degree distribution structure may include the 
typical behavior of its sub and superlinear relatives. See Remark 1.5 and 
Example 1.6 for more details and discussion. 

We also remark that the large deviations and other work are, with respect 
to the process, starting from either "small" or "large" initial configurations, 
that is, when the initial urn collection has o{n) balls (e.g., finite), or when 
the size of the collection is on order n, respectively. It appears these ini- 
tial configurations, which enter into all result statements, have not been 
considered before, in general. 

The main idea for the results is to extend a variational control prob- 
lem/weak convergence approach of Dupuis and Ellis (cf. [29]) to establish 
finite-dimensional LDPs in the time-dependent setting. Then, a projective 
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limit approach, and some analysis to identify the rate function, is used to 
obtain the infinite-dimensional LDP. For the LLN and power-law corollar- 
ies, a coupled system of ODEs, which governs the typical degree distribution 
evolution, is identified, and analyzed. 

To be concrete, we have focused upon models where the network is in- 
cremented by one urn and one ball each time, which include basic models. 
However, the methods here should be of use to analyze the large deviations 
of the degree structure in other combinatorial models with different incre- 
ment structure: for instance, the evolving graph model discussed in [16], 
Chapter 3, where at each time with probability p a new vertex is preferen- 
tially attached to an old one, and with probability 1 — p, an edge is added 
between two old nodes selected preferentially, and the BA graphs where, 
instead of only one vertex, m > 2 vertices are introduced and preferentially 
connected at each time; cf. [30], Chapter 4. 

1.1. Model. Let p{t) : [0, 1] [0, 1] and p{t) : [0, 1] [0, oo) be given func- 
tions. We define an urn configuration U as a finite collection of urns, each 
urn X (zU containing a nonnegative number of balls bx- We now specify an 
evolving array : < j < ra}„>i of urn configurations by the following 

time-dependent iterative scheme. In the nth row of the array: 

• Start at step 0, with a given initial urn configuration [/"(O). 

• At step J ' + 1 < n, to form a new urn configuration + 1), we first 
create and include a new urn with no ball. Then: 

- with probability p{j/n), we place a new ball in this urn; 

- with probability 1 —piJ/n), we place a new ball in one of the other urns 
X € f7"(j) with probability 

bx + /3{j/n) 

We will call, for urn x € U"'{j), the term bx + Pij/n) as the "weight" of the 
urn at time j in the nth row of the process. Let now and B^{j) = 

SxGC/"(j)^a; be the total number of urns and balls in U^{j), respectively. 
Then, the number of urns \U^{j) \ = |C/"(0)| + j and the total number of 
balls B^{j) = B"'{0) + j. Also, the total weight of the configuration at time 
j is 

^"(J):= E iby + Kj/n)) = il + Pij/n))j + B'\0) + l3ij/n)\U^m. 

The above urn scheme, as discussed in the Introduction, may be viewed in 
terms of the evolving degree structure in a preferential attachment random 
graph process with time-dependent selection function w{d;j, n) = d-\- f3{j /n). 
Here, the step of including a new empty urn and incrementing the number 
of balls in an old urn corresponds to an edge being placed between a new 
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node, with degree 1, and an old vertex in the existing graph whose degree 
is consequently incremented. In particular, when p and (3 are in particular 
forms, we recover the following models: 

(1) "Classical" BA process. When p{t) = and f3{t) = 1, the scheme is 
time-homogeneous. When the initial urn configuration consists of two empty 
urns, the probability of selecting an urn x with k>0 balls at time j > is 
(/c + l)/(2(j + 1)) , which matches the selection process in the evolution of the 
degree structure in the BA preferential attachment graph scheme at times 
j + 1 > 1 with selection function w{d) = d, as discussed in Example 1.1, where 
urns with A; > balls correspond to vertices with degree d = k + 1 >1. 

(2) "Offset" BA processes. When p{t) = and (3{t) =j3>Q, again the 
scheme is time- homogeneous, and urns with fc > balls correspond to ver- 
tices with degree k + 1 > 1. However, now the weight of an urn with k balls is 
A; + /?, in a sense "offset" from the classical BA scheme. Correspondingly, the 
urn selection scheme is the same as the growth process of the degree structure 
in the preferential attachment model with selection function w{d) = d + a 
with a = f3 — 1 as specified in Example 1.1. 

(3) CHJ model of Polya urns. When p(t) = p and /3(t) = /3 > 0, the evolu- 
tion of the number of urns of size fc > corresponds to a version of the r = 1 
CHJ model discussed in the Introduction. However, we note, in our model, 
an empty urn is added at each step with probability 1 —p, and these empty 
urns are kept track of in our scheme. When /3 = 0, the dynamics of urns 
of size /c > 1 is the r = 1 CHJ model since the empty urns have no weight, 
and once created, they cannot be selected to fill in later steps, and do not 
influence the structure of urns with A; > 1 balls. 

For n > 1, let Zf{j) be the number of urns in the reth row of the urn 
array process with f > balls at time < j < ?i and, for c? > 0, let Z^^^^j) 
denote the number of urns with more than d balls at time < j < n. These 
quantities satisfy 



d 



j;zr(i)+z,\i(i) = |[/"(o)i+j. 



d 



j^izru) + (d+ i)z,v(i) < i?"(o) +j. 



i=0 



Define now vectors in M'^+^ 



(0,1,0,..., 



where 



0), 





(1,0,. ..,0). 



For y = {yo,...,yd,yd+i) G 
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"'+2 and < i < (i + 1, denote 



[y]i ■=J2yi- 



1=0 



Note that 
(1.2) 



< [i% < 1 for < i < ci, 



d+i 



[fVi = l and 0<^(l-[f'^],)<l. 



i=0 



Consider now the "truncated" degree distribution 

{Z^'^U) := (Zo"(i), . . . , Z,\,(i))|0 < J < n}, 

where Z2_,,{j) = Ek>d+i ^W) = J + \Um\ " ELo ^fc (i)' which forms a 
discrete time Markov chain with initial state Z"'''^(0) corresponding to the 
initial urn configuration [/"(O) and one-step transition property, 

zM(j- + l)-Z-''^(j) 

B{j/n)ZS{j) 



'-0 ' 



fd 



with prob. p{j/n) + (1 —p{j/n)) 

for i = 0, 
with prob. (1 —p{j/n)) 
for 1 < i < d, 

wi.l,p.ob41-p(i/„))(l-£tM(!ffl 



We also define the "full" degree distribution 

{Z"'-(j) := (Z^U), Z2{j), . . .)|0 < J < n}, 
which is also a Markov chain on M°° with increments 
Z"'~(j + 1) -Z"'~(j) 

.PUMzsu) 



roo 



f oo 
'■i ' 



with prob. p{j/n) + (1 —p{j/n)) 

for i = 0, 
with prob. (1 —p{j/n)) 
for i > 1, 



i + /3(j7n))Zr(j) 



where = {0,1,0, ... ,0, ■■ ■) and = (1, 0, . . . , 0, -1, 1, 0, . . . , 0, . . .) with 
the "—1" being in the (i + l)th place. 
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We will assume throughout the following initial condition, which ensures 
a LLN at time t = 0. With respect to constants cf , c", c" > 0, for i>0, define 

i>0 

and 

j>0 

(LIM) For constants Cj, c, c > 0, we have 

Cj := lim c" and c:=limc"=> ici < oo. 

ntoo ntoo ^ — ' 

i>0 

Consequently, c := lim„|oo = X]j>o Q < oo. 

In the previous sentence, the c" limit follows from the uniform bound, 
Ei>A c? < Ei>o < cjA. Define also 

c'^:=^Ci and c'^ := (cq, . . . , c^). 

i>d+l 

We remark one can classify the initial configurations depending on when 
Cj = or when Q > for some i > 0. 

• (Small configuration) Cj = for any i > 0. Here, the initial urn configura- 
tions are small in that their size is o(n). This is the case when the initial 
configurations do not depend on n, for instance. 

• (Large configuration) Cj > for some i > 0. Here, the initial state is already 
a partly-developed configuration whose size is of order n. 

We also note, when the initial urn configurations correspond to initial 
tree configurations in the corresponding preferential attachment process, 
some restrictions in the values of q arise. One may verify that a graph with 
n vertices with degrees di, . . . , (i„ is a tree exactly when Y^=\ = 2(?t- — 1). 
Hence, since in the initial graph of the nth row, the number of vertices 
equals n'^f^^QC^, and the sum of degrees equals f^X]fc>o(^"'" ^)'^k (recall the 
correspondence between urn sizes and degrees discussed in the Introduction) , 
we have n X]fc>o(^ + ^)'^k ~ ^(^ ^k>o '^k~^)- (LIM), we have then c = c. 

In addition, we note (LIM) specifies an initial limiting degree distribu- 
tion which has full "weight" or in other words is not "condensed," that is, 
c = lim,„^oo = X^j>o ^Ci. See Remark 1.8, however, for comments when the 
initial distribution is "condensed," that is, c > X^j>o*Ci. 

Our results will be on the family of processes X"'*^ = {X"'''^(t) :0 <t <1} 
and X"'°° = {X"'°°(t) :0 < t < 1} obtained by linear interpolation of the 
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discrete-time Markov chains ^Z"''^(j) and ^Z"'°°(j), respectively. For 0< 
t < 1, let 

X"'"'(t) := iz"''^(lntj) + L^^J (Z"''^(M + 1) - Z'^'^(lntJ)), 
n ?i 

X"'°°(t) := iz"'°°([ntj) + L"^^ (Z"-°°([ntJ + 1) - Z"'°°( [ntj )). 
n n 

The trajectories X'^''^ lie in C([0, 1]; M"'"'"^), and are Lipschitz with constant 
at most 1, satisfying X"''^(0) = ^Z"''^(0). On the other hand, the infinite 
distribution X"'°° G HSo^d^' ^l'^)' considered with the product topology, 
where X"'°°(0) = iZ"'°°(0). In both cases, although X"''^(t) and X'^'~(i) are 
not necessarily probabilities because it is possible that we do not normalize 
by the total mass; they are, however, finite distributions. 

We now specify the assumptions on p(t) and P{t) used for the main re- 
sults. 

(ND) p and f3 are piecewise continuous and, for constants po, (3q and /3i, 
0<p(-)<Po<l and < /3o < /3(-) < A < oo. 

We discuss (ND) more in the remark after Theorem 1.2. 
We note, throughout the article, that we use conventions 

OlogO = 0/0 = • ±oo = l/oo = 0, 
(1.3) ±l/0 = ±oo and 

E[X-k]= [ XdP. 

J A 

1.2. Results. We now recall the statement of a large deviation principle 
(LDP). A sequence {^"} of random variables taking values in a complete 
separable metric space V satisfies the LDP with rate n and good rate func- 
tion J:V ^ [0, oo] if for each M <oo, the level set {x G V|J(x) < M} is a 
compact subset of V, that is, J has compact level sets, and if the following 
two conditions hold: 

(i) Large deviation upper bound. For each closed subset F of V, 

limsup-logP{X" G F} < - inf J{x). 

(ii) Large deviation lower bound. For each open subset G of V, 



liminf-logP{X" G G} > - inf J{x). 

rn-oo n xeG 
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For d > 0, we now state the LDP for {X"''^(t)|0 < t < 1}. Define the func- 
tion Id : C([0, l];M'^+2) ^ [0, oo] given by 




1 - imh 

-P[t))(^l+p(t))t+c+cm 

log '-gto(l;10W].) 

where (p{0) = c'^, > is Lipschitz with constant 1 such that < [^{t)]i < 1 

for < i < d, ES'o = 1' Eto(l - = Eto i^iii) < 1 for almost 

all t, and the integral converges; otherwise, Idif) = oo. It will turn out that 
Id is convex and is a good rate function. 

To explain the last condition in the definition of Id, note that ipd+i{t) 
represents the fraction of urns with size at least d+1, so that {d+ l)ipd+i{t) 
is the truncated fraction of balls in these urns. Since the process incre- 
ments by one ball at each step, it makes sense to specify Yli=o(^ ~ — 
Y^i=o i^i{t) < 1 or that Y^itl i'fi{t) <t + ciiId{(p)<oo. 

The rate function can be understood as follows: in order for X"'"^ to 
deviate to ip, at time t, the process should behave as if the increment prob- 
abilities Vi of if are such that the mean Yli=o^i^f + f^d+if^+i ~ V'- 
proof of Theorem 1.2, we show Vi = 1 — [ip]i for <i < d and Vd+i = 1 — 
Sj=o(-l ~ But, the natural evolution increment probabilities Ui, given 
the process is in state ip{t), are uq = p(t) + {l-p{t))jj^:^^^^^^^^, Uj = (1- 

Pirjj(i+/3(t))i+c+c/3(t) tor i S i S a ana Ud+i - [i PirjjU (i+/3(t))t+c+c/3(t) 
Then Id is time integral of the relative entropies of these two increment prob- 
ability measures. 

Recall, for probability measures /i and z/, that the relative entropy of 
with respect to is defined as 

otherwise. 

Theorem 1.2 (Finite-dimensional LDP). The C([0, 1];W^-^^) -valued se- 
quence {X"'"^} satisfies an LDP with rate n and convex, good rate func- 
tion Id- 
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Remark 1.3. We now make comments on the underlying assumption 
(ND) and obtain the rate function at the fixed time t = l. 

(A) The assumption (ND) specifies that the process considered is "non- 
degenerate" in some sense. (ND) does not cover some "boundary" cases, for 
instance, when p{t) = 1, the process is deterministic in that at each time, 
one places a new ball in a new urn. Also, when /3(t) = 0, urns without a ball 
have no weight; and, if in addition p{t) = 0, then all new balls are placed 
into urns in the initial configuration. Although an LDP should hold in these 
and other "less degenerate" cases, the form of the rate function may differ 
in that some increments may not be possible. 

On the other hand, assumption (ND) is natural with respect to the con- 
vergence estimates needed for the proof of the lower bound in the LDP. 
However, we point out the LDP upper bound holds without any of the 
boundedness assumptions on p(-) and /3(-) in (ND). 

Formally, when f3{t) = oo, this is the case of "uniform," as opposed to 
preferential, selection of urns. The limit lim^|oo corresponds to the rate 
function for this type of dynamic. 

(B) One recovers the LDP at a fixed time, say t = 1, by the contrac- 
tion principle with respect to continuous function F: C([0, 1];M^"^^) M"'"''^ 
defined by F{ip) = (p{l), so that F(X'^'^) = X"'^(l) = ^Z"''^(n). Then The- 
orem 1.2 implies the LDP for ^Z"''^(n) with rate function given by the 
variational expression K{x) = inf{/rf(</?)|(/j(0) = c'^,ip{l) = x} which might 
be evaluated numerically; cf. [13] for calculations when d = 0. 

We now extend the finite-dimensional LDP results to the infinite-dimen- 
sional case {d = oo). Define for ^ £ nSo^d^' M't^) the function 



1^(0= /' lim 



1 - am 



E(i-[M.)].o. ^-^to(i- [a»)i.) 



dt 



^ »=0 ^ -P{t)){^ - (i+^(t))t+c+c/3(t)i- 

where ^i(O) = q, ^j(t) > is Lipschitz with constant 1, < [^(t)]i < 1 for 

i>o,i Er=o m = 1 and iim,Eto ut) + {d+ i)(i - mu)] = E£o(i - 

[^(t)]j) < 1 for almost all t, and the integral converges; otherwise = oo. 

It will turn out through a projective limit approach (cf. [20], Section 4.6) that 
I°° is well defined, convex and a good rate function, and that the integrand 
limit exists because the term in square brackets is increasing in d. 
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Theorem 1.4 (Infinite-dimensional LDP). The l\ZoC{[0, l];R)-valued 
sequence {X'^'°°} satisfies an LDP with rate n and convex, good rate func- 
tion . 

Remark 1.5. From the result, degree distributions, not fully supported 
on the nonnegative integers, that is, when X]j>o < t + c or, in other 
words, when the distribution specifies a positive fraction of urns with an infi- 
nite number of balls, cannot be achieved with finite cost in the evolution pro- 
cess. This stabilization of the "mass" is understood as follows. The fraction 
of urns with size larger than A at time \nt\ is bounded in terms of the frac- 
tion of bahs in the system: Efc>A ^fc (U*J)/" ^ ^""^ I]fc>o ^^fc (L^*J)/^ ^ 
A~^{ [nt\ /n + c^) < A~^{1 + 2c) for all large n. Hence, for all realizations of 
the process, the fraction of infinite sized urns vanishes. 

On the other hand, it seems some fraction of the total "weight" can indeed 
be lost in the evolution process with finite rate, that is, it may be possible to 
achieve a degree distribution at a time < t < 1 such that Yli=o <t + c 
although pre-limit ^"^QiZ^djitD/n = [nt\/n + c^. The interpretation is 
that it is possible to put a positive fraction of the balls into a few very 
large urns with finite cost, a sort of "condensation" effect noticed in the 
limiting evolution when the selection function is superlinear as mentioned 
in Example 1.1. 

The last term in the integrand of the rate function, corresponding to the 
increment f^^^, measures the cost of choosing urns with very large size. 
In the d^ oo limit, this last term may be viewed as the cost of "escape" 
of weight from urns with bounded size, or, in other words, the cost of the 
increment "(1, 0, . . . , 0, . . .)" which corresponds to a new empty urn being 
included and very large sized urns being incremented. Some "condensed" 
finite rate evolutions are discussed in Example 1.6. 

However, on the other hand, this type of "weight" loss or "condensation" 
cannot happen in the typical evolution — see Corollary 1.7. 

Example 1.6. Consider the "classical" BA model which follows the 
evolution of a random graph with preferential attachment selection function 
w{d) = d, noted in Example 1.1 and Section 1.1, which corresponds to the 
urn system when f3{t) = 1 and p{t) = 0. Suppose that the initial configura- 
tions satisfy Cj = for all i > 0. 

We now compute the cost of distributions in form ^(t) = tj where 7 = 
{"fi'.iy 0) where constants 7i > are such that 

^7i = l and ^i7, = ^(l-[7],)<l. 

i>0 i>0 i>0 

Since, ^(t) is linear in t, calculation of the rate /°°(C) simplifies considerably, 
and one evaluates the limit of the last term in the integrand of /°° (^) as the 



14 



J. CHOI AND S. SETHURAMAN 



time-independent quantity, 



i-(Eto(^ + i)e^W)/(2t) 



i>0 



which gives the cost of the "increment (1, 0, . . . , 0, . . .)" when the dynamics 
attaches new vertices to very large hubs or places balls into already very 
large urns. 

This cost is positive if X]j>o < !> and, as discussed in the remark above, 
corresponds to the cost of forming urns/nodes with very large size/degree 
in the evolution process, a "condensation" effect. It follows then 



In the case 70 = 1 and 7i = for i > 1, one observes /°°(^) = log 2, and 
one can associate a graph evolution to achieve this degree or size distribu- 
tion. For instance, one may grow a "star" tree configuration where all new 
vertices connect to the same vertex, or all balls are put in the same urn. If 
initially, there are only two vertices with degree 1 or two empty urns, then 
the "star" configuration at the nth step has probability 2~" of occurring. 
As the degree/size structure at time n consists of n leaves/empty urns and 
one vertex with degree n or one urn with size n — 1, one observes the LLN 
limit for the degree/size sequence is ^(t) = tj, from which the rate evaluation 
follows. 

As discussed in Example 1.1, this "condensed" configuration is the limit 
tree with respect to superlinear selection function w{d) = dJ' for r > 2. More- 
over, as noted in the Introduction, all preferential attachment evolutions 
with respect to superlinear selection function w{d) = d"^ for r > 1 lead to 
degree distribution 7, that is, EZi{n)/n — > 7j, where 70 = 1 and 7i = for 
i > 1. 

From formula (1.4), when 7 is supported only on a finite number of indices, 
one sees that < 00 exactly when there exists z* > such that 7i > for 

i <i* . In particular, the "straight road" evolution, leading to trees where all 
nodes have degree 2, except for two leaves, or urn configurations consisting 
of single ball urns except for two empty urns, has infinite cost: start with two 
vertices with degree 1 or two empty urns. At step j + 1, connect a new vertex 
to one of the two leaves, or add an empty urn and place a ball in one of the 
two empty urns in the configuration formed at step j. This configuration at 
time n has probability 1/n! of occurring, and in the LLN limit corresponds 
to ^(t) = ^7, where 70 = 0, 71 = 1 and 7^ = for i > 2, for which /°°(^) = 00. 



(1.4) 
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Even when no weight escapes, that is, J2i>o^'yi = it may be noted that 
deviations to nonpower law urn size paths ^ are possible with finite rate. 
For instance, when 7^ = 2-(^+^) for i > 0, = - Ei>o i^tt log When 

li = ^(^ + 1) for i >0 and q in form of the stretched exponential in (1.1), 
the LLN limit for the degree distribution with respect to sublinear selection 
preferential attachment, a calculation verifies that X]j>o^7« = 1 and also 
< 00. 

We now turn to the LLN behavior which corresponds to the "zero-cost" 
trajectory. Consider the system of ODEs for Lp'^ = ip, with initial condition 
V?(0) =0^^: 

/3(t)(^o(t) 



^o{t) = l-p{t)-{l-p{t)) 
p^i{t)=p{t) + {l-p{t)) 
-{l-p{t)) 



l + /3(t))t + c + c/3(t) 



(L5) 

^i{t) = {l-p{t)) 



{l + p{t))t + c + cl3{ty 
(l + /3(t))t + c + c/3(t) 



(1-P(t))-41#1M%^ for2<.<d. 



i=0 

Recall that a "Caratheodory" solution is an absolutely continuous func- 
tion satisfying the ODEs a. a. t, and the initial condition, or equivalently 
a function satisfying the integral equation associated to the ODEs. One 
can readily integrate ODEs (1.5), and find a Caratheodory solution C'^{t) = 
{Co{t), • • • , Cd{t),Cd+i{t)) [see formula (4.1)], which is unique from the follow- 
ing theorem. One extends to "d = 00" setting by defining 

r(t) := {Co{t),...,Cd{t),...). 

We now state a LLN for X"'*^ and X"'°° as a consequence of the LDP up- 
per bound. As remarked in the Introduction, this LLN may also be obtained 
by rate equation formulations as in [36] and [30], Chapter 4. 

Corollary 1.7 (LLN). For d>0, is the unique Caratheodory solu- 
tion to ODEs (1-5) with the initial condition (/^(O) = c'^, and also IdiC^) = 0. 
Then, in the sup topology on C{[Q,l\;W^+'^), X"''^(-) ^ C'^(-) a.s. 
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timet = 0.01 timet = 0.1 time t = 1 

logaco) iog(4(0) iog(fi(0) 




Fig. 1. The thick curves are the (numerical) LLN ODE paths at times 4 = 0.01,0.1,1 
with p{t) = 0, I3{t) = 8 for t < 0.01, 1 for t > 0.01 and Ck = 0. Dashed lines are straight 
lines with slope —3 and —10. The plots use log-log scales. 



As a consequence, we have in the product topology on Y\'^qC{[0,1];M) 
that X"'~(-)^C~(-)- Moreover, EZodit) =t + c and EZo^dit) = t + c, 
and hence no "weight" is lost in the LLN limit. 

Remark 1.8. The last equality, J2i>o^^i(^) = t + c, requires the condi- 
tion in (LIM) that the initial scaled degree distribution is not "condensed," 
that is, c = lim^^oo = X]j>o When the initial distribution is "con- 
densed," that is, a strict Fatou limit c = lim„-|-oo c" > X]j>o^Ci occurs, the 
large deviation results Theorems 1.2, 1.4 and Corollary 1.7 (except for 
the last equality) still hold with the same notation and proofs. However, 
one can show by similar arguments as for the proof of the last equality 
in Corollary 1.7 that the LLN trajectory (^°° will now be "condensed," 
that is, s(t) = X]j>o^Ci(0 < t + c for t > 0. Moreover, for a constant C = 
C{c,c,pq, f3i, (Sq) > 0, one can see for all large t that 

c(c-Y, ici) <t + c-sit)<C-'(d-Y^ . 

^ i>0 ^ ^ i>0 ^ 

We now consider the "scale- freeness" of Although it seems difficult 
to control each Q, nevertheless has "power law" behavior, in terms of 
bounds on [C°°]i- In general, it appears can interpolate between the 
bounds (cf. Figure 1; as a curiosity, we note a figure with a similar "bend" 
is found in [33] with respect to Facebook social network data). 

Corollary 1.9 (Power law). Assume < Pmin < ?{•) < Po =■ Pmax < 1; 
and < /?o =: /3min < /3(-) < /3max '■= Then, is bounded between two 
power laws: 
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For small configurations, for example, Ck = 0, we have, for i>0 and t>0, 

W]^t<[^m<bl]^t■ 

For large configurations, for example, > for some k >0, we have, for 
i>0, 

[v'ut + o(i)) < [rm < [vw + o(i)) as n oo. 

Here, with respect to positive constants C, C depending on p and f3, 

C 

and 

(J 

il + {l + /3max)/{l-pmax) ^ ^^^^^ ' 

The outline of the paper is that in Sections 2 and 3, we prove the finite 
and infinite-dimensional LDPs, Theorems 1.2 and 1.4. In Section 4, we prove 
the law of large numbers (Corollary 1.7). Finally, in Section 5, we discuss 
power-law behavior (Corollary 1.9). 

2. Proof of Theorem 1.2. We follow the method and notation of Dupuis 
and Ellis in [29]; see also [52]. Some steps are similar to those in [13] where 
the "leaves" in a more simplified graph scheme are considered. However, as 
many things differ in our model, in the upper bound, and especially the 
lower bound proof, we present the full argument. 

We now fix < d < oo and equip with the Li -norm denoted by | • |. 
Recall, from assumption (LIM), 

c"'"^ = (c?}, . . . , c^, c"'"^) := -Z"'°'(0) ^ c'^, 

n 

where c"'*^ = '}2i>d+i Denote 

i{n,t):={pn{t),Pn{t),an{t)), 

where 

Pn{t) :=p{[nt\/n), /3„(t) :=/3(M/n), 

<Tn{t) := -s^{[nt\) = (1 + Pn{t))^ + c" + c"/3„(t). 
n n 

Let 

a{t) := (l + /3(t))t + c + c/3(t), 
m :=(p(t),/3(t),a(t)). 
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We note that, as n — )• oo, and p(t) and I3(t) are piecewise continuous, 

i{n,t)^ ^{t) for almost all t G [0, 1] . 

In the remainder of the section, when the context is clear, we often drop 
the superscript d to save on notation. Recall 

X-(j) := -Z^'''ij), 
n 

X"(0) = c"-'^ and X"(j + 1) = X"(i) + ^y^^^j^U), where 
y^ii) has distribution p^, ., ^ . 

Here, for x = {xq, . . . ,Xd,Xd+i) € M'^"'"^ such that Xj > for < i < d + 
1, numbers p' G [0,1] and (3', a' > such that Ef=o(^ + ^l^i < ^r', and 



P(p',/3V'),x(^) :=(p' + (l-p')^)'^fo(^) 



z — / 0-' 

We note when a' = and x = (0, . . . , 0), by convention 0/0 = and 
P(p',/3',0),x(^) ■=pSfM) + (1 -P')'^f.+ i(^)- 

From (1.2) and (LIM), for ^ > 0, the paths X"(t) = X"''^(t), for all large n, 
belong to 

Td,A ■■= f^^p £ C{[0,l];R'^+^)\\ip{0) - c'^l <A,ipi is Lipschitz 
(2.1) with bound 1, < [ip{t)]i < 1 for < i < d + 1, and 

d+l d+1 d ^ 

i=0 1=0 i=0 ) 

Here, we equip C([0, 1];M'^"'"^) with the supremum norm. 

Let h : C([0, 1]; M''^^) — ?> M be a bounded continuous function. Let also 

W := -^log£;{exp[-n/i(X'")]}. 
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To prove Theorem 1.2, we need to establish Laplace principle upper and 
lower bounds (cf. [29], Section 1.2), namely upper bound 

liminfW^"> inf {Id{ip) + h{ip)} 

for a good rate function 1^, and lower bound 

limsup VF" < inf {/^(v?) + h{ip)}. 

n^oo ^eC{[0,l];R'i+2) 

Given X"(0) = c"-'^, define, for l<j<n, that 
H^"(j,{xi,...,x,}) 

:= logE{exp[-n/i(X")] |X"(1) = xi, . . . , X-(j) = x,} 

and 

W"" := VF"(0, 0) = --logS{exp[-n/i(X")]}. 
n 

The Dupuis-Ellis method stems from the following discussion. From the 
Markov property, for 1 < j < n — 1, 



e 



-nM/"(j,{xi,...,x,}) 

= i^le-"'^^^") |X"(1) = XI, . . . , X"(i) = X,} 

= i^{i?{e-"'^(^") |X"(1), . . . , X"(j + 1)}|X"(1) = XI, . . . , X"(j) = X,} 
= i^{e-"^"{i+i,{X"(i),...,xn.),xni+i)})|x"(i) = XI, . . . ,X"(j) = X,-} 

nM/"(j+l,{xi,...,x,,x,+y/n}) N 



Recall the definition of relative entropy near Theorem 1.2. Then, by the 
variational formula for relative entropy (cf. [29], Proposition 1.4.2), for 1 < 

j <n-l, 

T^"(j,{xi,...,x,}) 



We also have a terminal condition W'^{n, {xi, . . . ,x„}) = /i(x(-)), where x(-) 
is the linear interpolated path connecting {(j'/n,Xj)}o<j<n- 

We may understand these dynamic programming equations and terminal 
conditions in terms of a particular stochastic control problem. Define: 
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(i) Lj = (M''+^)-^, the state space on which W^{j, ■) is defined; 

(ii) lA = 'P(]R'^'^^), where V{B) is the space of probabihties on B, is the 
control space on which the infimum is taken; 

(iii) for j = 0, . . . , n — 1, "control" Vj'{dy) = 7;"(dy |xo, . . . , Xj) which is a 
stochastic kernel on R'^+^ given (]R'^+^)-' ; 

(iv) {X"'(j);0 <j< n}, the "controlled" process which is the adapted 
path satisfying_X"(0) = c"'*^ and X"(j + 1) = X"(j) + ^Y"(j) for < j < 
n — 1, where Y"(j), given (X"(0), . . . , X"(j)), has distribution v'j{-) [i.e., 
P{Y"(j) G dy|X"(0), . . . ,X-(j)} := v^^ {dy\X.^ (0) , . . . ,X"(i))] and X"(.) is 
the piecewise linear interpolation of (j/n,X"(j)); 

(v) "running costs" Cj{v) = ^R{v\\p) for v G P(M'^+2); and 

(vi) "terminal cost" equals to the function h. 

Also, define, for < j <n — 1, the minimal cost function 

y"(j,{xi,...,x,}) 

where ff'(-) = f"(-|X"'(0), . . . ,X"'(z)), and the infimum is taken over all con- 
trol sequences {v^}. Here, -E'j,xi,...,xj denotes expectation, with respect to 
the adapted process X"'(-) associated to {vf}, conditioned on X"(l) = 
xi, . . . ,X"(j) = Xj. The boundary conditions are y"(n, {xi, . . . ,x„}) = /i(x(-)) 
and 

(2.2) V- :=y"(O,0) = MEf^lj2R{v^^ 

It turns out that {y"(j, {xi, . . . , Xj}) : < j < n} also satisfies the dy- 
namic programming equations and terminal condition, and since these equa- 
tions have unique solutions (cf. [29], Section 3.2), we may conclude by [29], 
Corollary 5.2.1, that 

W"" = -- log^{exp[-n/i(X"(-))]} = V". 
n 

2.1. Upper bound. To prove the upper bound, it will be helpful to put 
the controls {Vj'} into continuous-time paths. Let v'^{dy\t) := Vj'{dy) for 
t € [j/n, {j + l)/n), j = 0, . . . ,n — 1, and v"'{dy\l) := t'".!. Define 

t;"(Ax5):= / v''{A\t)dt 
Jb 

for Borel A C and S C [0, 1]. Also define the piecewise constant path 
X"(t) := X"(j) for t G [j/n, (j + l)/n), 0<j<n-l, and X"(l) := X"(n - 1). 
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Then 

= = i^f ^K(-|*)ll/'c>,t),xn*)) + ^(^")}- 

Given is supported on K := {fo, fi, • • • , fd+i}i if {vl} is not supported on 
K, then i^fz;"!!/)^ ) = oo. Since IF"! < ||/i||oo < co and -ftT C ]R'^+^ is compact, 
for each n, there is {vj} supported on K and corresponding v'^[dy x dt) = 
v"-{dy\t) X dt such that, for e > 0, 

(2.3) W^ + e = V^ + e>El^j'^ ^K(-|*)IIPf(n,i),X"w) dt + /i(X")| . 

Recall that X"'(-) takes values in Td A. Since T^^a is compact, by applica- 
tions of the Ascoli-Arzela theorem, and {t^"} is tight, by Prokhorov's theo- 
rem, given any subsequence of {?;",X"}, there is a further subsubsequence, 
a probability space {Cl,^,P), a stochastic kernel u on i^T x [0, 1] given and 
a random variable X mapping into T^^a such that the subsubsequence 
converges in distribution to ,X). In particular, since X"(0) = c"'*^ as 
n oo, we have X [cf. (2.1)] belongs to 

Trf := ^d,o those functions such that (/^(O) = c"^. 

Then, [29], Lemma 3.3.1, shows that v \s a subsequential weak limit of t;", 
and there exists a stochastic kernel v{dy\t,uj) on K given [0, 1] x Q such that 
P-a.s. for oj (^Q, 

v{A X B\oj) = / v{A\t,u})dt. 
Jb 

Now, the same proof given for [29], Lemma 5.3.5, shows that (v", X", X") 
has a subsequential weak limit (u,X,X), where the last coordinate is with 
respect to Skorokhod space L'([0,l];M'^+2), and P-a.s. for t G [0,1] 

X(t) = / yv{dy xds)= [ ( [ yv{dy\s)] ds, 

JRd+2x[0,i] Jo \Jk / 

±it) = [ yv{dy\t). 
Jk 

By Skorokhod's representation theorem, we may take that (i;", X", X") con- 
verges to (f,X,X) a.s. In particular, X" ^> X uniformly a.s., and as X 
is continuous, it follows that also X" — )■ X uniformly a.s.; cf. [29], Theo- 
rem A. 6. 5. 

Let A denote Lebesgue measure on [0, 1] and p x X product measure on 
K X [0,1]. Then [29], Lemma 1.4.3(f), yields 
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We now evaluate the limit inferior of W"' using formula (2.3), along a 
subsequence as above: 



liminf + . > liminf ^{ j[ ^(^"(- WII/'fM),X"w) + MX") 

= lim^inf^{i?(^;"(-|t) x A(dt)||/5^-(„ x"(t) >< ^(dt)) + /i(X")} 
>E{R{v{-\t) X A((it)||p^-(,) x(t) X Kdt)) + KX)} 



E 



Note that we used Fatou's lemma in the second inequality, observing (i)-(iv). 

(i) v"' {dy\dt) X X{dt) -^v{dy\dt) x X{dt) a.s. as u " =^ v a.s.; 

(^^) Pi{n,t),±"{t) ^ PmS{t) ^ ^ ^^^^ ^-^^ * ^ ^^^^ ^ 

X(t) uniformly on [0, 1] a.s.; 

(iii) liminf„_^oo-R(w"(dy|dt) x Kdt)\\p^(^^^^^^^„f^^^ x X{dt)) > R{v{dy\dt) x 
A((it)||p|'^^^ -j^j-j^ X X(dt)) a.s. as is lower semi-continuous; 

(iv) /i(X") ^ /i(X) h is continuous and X" — )• X uniformly on 
[0, 1] a.s. 

By [29], Lemma 3.3.3(c), 

^«\i)\\PmMt) ) - ^ (jit)Mt)J^ zv{dz\t)^ , 

where 

L(e(t),x,y) : = sup|(0,y) -log^exp(0,z)p^-(^),^((iz)|0GR'^+2| 

= inf|i?(z.(.|t)||p^-(^) J|z.(.|t)e W,y^zz.(dz|t)=y|. 

We note, in this definition, the infimum is attained at some z/q € V{K) 
as the relative entropy is convex and lower semicontinuous; cf. [29], Lem- 
ma 1.4.3(b). Since J" zv{d7.\t) =X(t), we have 

liminfy">^| ! L(^"'(t),X(t),X(t))dt + /i(X)l. 

n^oo [Jo J 

As X € Frf, we have 

liminf1/"> inf | /" L(^"'(t), (^(t), v3(t)) + /i((^)l . 
n^oo yperd [Jo J 
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For ip G r^, we can evaluate a unique minimizer z^o('|i) in the definition of 
L(^(t),(/5(t),vi(t)): recall tliat \p(t)\i := I];=o'^K*)- Then, as Ef=o fi^o(fj|*) = 
{(po{t), . . . , a calculation gives 

d / d \ 

(2.4) Mm\t) = 5^(1 - [9i(i)].)5f, + - d df,^,. 

i=0 \i=0 / 

Hence, 

L{i{t),^{t),^{t)) 
=R{Mmt)\\Pi^t),^^t)) 

(2.5) =(l-[,3(t)]o)lor '-t'^^'^]" 



1 - m)]i 



V & V (i-P(t))(i- g^~g )- 



interpreted under our conventions (1.3). 
Finally, define 



r((^):= / L{({t),ip{t),ipit))dt, 
Jo 



when G Frf, and Id{^) = oo otherwise. Since L is convex, is convex. Also 
Id has compact level sets by the proof of [29], Proposition 6.2.4, and so is 
a good rate function. Hence, the Laplace principle upper bound holds with 
respect to Id- 

We will need the following result for the proof of the lower bound in the 
next section. 

Lemma 2.1. Let l{t) = et + c*^ he a linear function, where e = (eo,ei, . . . , 
ed+i) is such that Cj > for i > 0, '^Z'iZo^i — ^''^d X]^=o < 1- Then, 
/rf(£(t))<oo. 

Proof. Noting j:to(l " NO = Ef=o < 1> explicitly 

1 - No 



idm)= I (1- No) log 

Jo 



24 J. CHOI AND S. SETHURAMAN 

+ - [e],) log 

+ f 1 _ V(i - H,)) log '-^toa-i-i-) 

I t: "V ^i-pw)a- ^g^;;~-' ) 

is bounded under the bounds on p, f3 in assumption (ND). □ 

2.2. Lower bound. Fix /i : C([0, 1];M'^^^) —)> M, a bounded, continuous func- 
tion, and if* € Td such that Id{^*) < oo. To show the lower bound, it suffices 
to prove, for each e > 0, that 

(2.6) limsupV <Id{^*) + h{ip*) + 8e. 

n— >oo 

The main idea of the argument is to construct from ip* a sequence of control 
measures suitable to evaluate formulas for V^. 

Note only in this "lower bound" subsection, to make several expressions 
simpler, we often take Cd+i := c'^- 

2.2.1. Step 1: Convex combination and regularization. Rather than work 
directly with 93*, we consider a convex combination of paths with better 
regularity: for < < 1, let 

where £{t) = et + c'^ is a linear function such that e satisfies the assumptions 
of Lemma 2.1, say e = (i, ^, . . . , 

Lemma 2.2. As 9^0, we have 

W)I^O and \h{^e)-h{^*)\^0. 

Proof. By convexity of Id, and finiteness of Id{£{t)) from Lemma 2.1, 

ld{^e)<{l-0)ld{^*) + eid{(). 

On the other hand, since \'fe{t) — ip*(t)\ = \ j^^^e — ^*){s) ds\ < 2t9{d + 2), 
we have \\ifg — v?* ||oo < 26'(d + 2) J, 0, by lower semi-continuity of Id, we have 



liminf Id{(pe) > Id{<^*)- 

Also, as h is continuous, we have that \h{ipQ) — h{ip*)\ — t- 0. □ 

Now, fix > such that 

Id[^e)<Id{.'P*) + £ and h{ifo)<h{ip*)+e. 
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(2.7) 
where 



(feis)ds 



for t G [i/K, {i + 1)/k), < i < k - 1, and 7^(1) = ^^{1 - 1/k). Note that 
V'k S r^, and on [i/n, {i + for < i < k — 1, ipnit) equals the constant 

vector 7K(^/K). In particular, ip^ is a step function. 

Lemma 2.3. For <i < d + 1 and <t <1, 
(2.8) V'K,*(t)>^(e*t + Ci), 



(2.9) 



i=0 



i=0 



Proof. These are properties of (pg inherited from properties of ip*,i € 
Td, which are preserved with respect to (2.7). Indeed, for each < i < d+ 1, 

1pK.,i(t) = ipg^i{[tK\/K) + {tK - ltK\){ipg^i{{[tK\ + 1) / k) - ipg^i{[tK\ / k)) 

>e{eit + Ci). 

Last, (2.9) follows: noting that - [e]i) = X^f=o iej = 1 - e^+i. 



i=0 



{1+1)/k 
1/k 



- [ d d 

{i-e)Y^{i-[^*{s)\,) + eY,{^-m],) 

i=0 i=0 

<l_0 + ^^(l_[e],) = l_^erf+i. 

1=0 

Lemma 2.4. For large enough k, we have 
(2.10) h{^^)<h{^*) + 2e and h{i^^) < h{v*) + 2e. 

Proof. Since 

lim sup |996)(t) -'(/'«(*) I =0, 



ds 



□ 
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the inequality with respect to h follows from continuity of h and choosing k 
in terms of 6. We also note, by absolute continuity of (fe, that a.s. in t, 

ipnit) ='jK{t) = K ipe{s) ds ^ ipe{t) as/ttoo. 

J [i/tJ / K 

Then, by the form of L [cf. (2.5)], bounds in Lemma 2.3 and piecewise 
continuity and bounds on p, f3 in assumption (ND), we have, as Kfoo, that 

L{i{t),il;^{t),i;^{t))^ Licit), Mt),Mt)) for almost alHG [0,1]. 

Also, we can bound L(^(t),^/;«;(i), V'k(^)) as follows: first, using xlogx < 
for < X < 1, bound that 



-|:a-[«.)i.).o.((i-P(*))^,f«^ 



i-^ii-[i^,it)]A 

. i=0 / 



(l + /3(t))t + c + c/3(t) 



Now, as < [ipK\i < 1 and < Yli=oi^ ~ [^k]«) ^ Ij we have the further 
upperbound, using Lemma 2.3, 

/3(t)0(eot + co) 



\og(pit) + il-pit))- 



(l + /3(t))t + c + c/3(t) 
.... ii + mWie,t + c,) \ 



^l + /5(t))t + c + c/3(t) 

which is integrable on [0, 1] given the bounds on p,f3 in assumption (ND). 

By dominated convergence, we obtain lim^ /^(V're) = Idi'^e)-, and therefore 
the other inequality with respect to J^. □ 



Let now k be such that (2.10) holds. Finally, we modify ifj^ on the interval 
[0, 5], for a small enough 5 > to be chosen later. 
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Define 

d 

(2.11) ti:=6-J2i^+[c%-[Mm 

i=i 

for < i < d, and t^+i := 5; set also t^i := and = td+i - Let also 

(2.12) r{t)= fl*{s)ds + c\ 

Jo 

where 

rfd+i, whenO<i<to, 
= < fj, when ti <t < ti+1,0 <i<d, 

l7«;(t), whent>J. 

Note 7* may not be defined at some endpoints as possibly ti = tj+i for 
some i. 

By inspection, tp* G r^;. Also, 'ip*{t) = f^+i when < t < to and V'*(*) = fj 
when ti <t < tj+i for < i < d. Moreover, we have the following properties. 

Lemma 2.5. We have ^*((5) = V'k(^) a^-c^ *o > ^e^+i^. ^/so, 
V'S(0 = i + co, V';(t) = Cj /or l<i<d+l, 
when <t <tQ, and 

V'o(*) — ^^d+i*^ + Co w/ien to < ^ < ill 

ipi{t) > 6{ei5 + Cj) when t j < t < tj+i and 1 < z < d. 

Proof. The lower bound for to follows from the integration of both sides 
in (2.9) and the definition of to. Now, we note that ipoit) = if to < t < ti, 
and 1 otherwise. Also, note that for 1 < i < d + 1, ^*(t) = 1 if ti-i <t <ti, 
'ip*{t) = -1 if ti < t < tj+i and ip*{t) = otherwise. Thus, noting (2.11), 

V'o(^) = / lU^) ds + Co = 5 - (ti - to) + Co = V'k,o(5) 
Jo 

and, for 1 < i < d + 1, 

i^ii^) = / 7j*(s) ds + Ci = {ti - ti-i) - (ti+i - ti) + Ci = V'K,j(5), 
Jo 

which proves that ip*{S) = '0ft(<^')- Since "00(0 nondecreasing, for t > to, 
V'o(*) > V'S(*o) = to + Co > Ocd+iS + Co. For 1 < i < d, for t^ < t < tj+i, V'*(i) 
decreases to its final value 4'K,i{5) > 9{ei6 + Cj) by (2.8). □ 
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2.2.2. Step 2: More properties of ip* . We now show the rate of ij:* up to 
time 5 does not contribute too much. 

Lemma 2.6. For small enough 5 > 0, 

L{i{t),i^*{t),ip*{t))dt<e and HV'* - V'kIU < £• 

In particular, h{ip*) < h{ip*) + 3e and Idii^*) < Id{v*) + 3e. 

Proof. Write, for < t < 5, as L{i{t),r{t),r{t)) = R{6f,J\p^^^^^^^^^^)x 



^.C())l(ii <t< ti+i), 



Lm,rit),r{t)) 

By Lemma 2.5 and the bounds on p,/3 in assumption (ND), this expression 
is integrable for <t < 6. (It would be bounded unless c*^ = and c 7^ 0, 
in which case the first term in the expression involves — logt.) Hence, the 
first statement follows for small 5 > 0. Also, the second statement holds 
as IIV'* — V'kIIoo = supo<t<5 — V'kI < 26{d + 2). The last statement is a 
consequence now of (2.10). □ 

We will take 6 > small enough so that the bounds in the above lemma 
hold. 



1 

T) ^ ^ 



n) - c'' 



n ■ 
1=0 



0. 



Lemma 2.7. We have 

(2.13) lim sup 

n^ooo<j<„ 

Also, for j > [Sn\ and < i < d + 1, 

(2.14) ^E^nV-) + Q>^(^ + c.). 

1=0 ^ ^ 

Proof. Since -0* is piecewise constant, when Z/n < ,s < (Z + l)/??,, |^*(s) — 
'(/'*(//?i)| ^ for at most k subintervals [cf. (2.7) and (2.12)], and is also 
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bounded by 2{d + 2). Hence, 
1 

r{j/n)--Y,r{i/n)-c' 



E / {r{s)-r{i/n))ds 

1=0 1=0 -^'Z" 

^ 2(d + 2) 

< K. 

n 

The last statement follows from (2.8). □ 

2.2.3. Step 3: Admissible control measures and convergence. We now 
build a sequence of controls based on . Define z/q = I'oi.i^* {j / / n) using 
(2.4), and 



?;"((iy;xo,...,Xj) 

' T^o{'4'*{3/n)\j/n), 



Pi: 



when < J < \_6n\ 
or when j > [5n] 

and Xj^j > -^{eiS + Cj) 

for 0<i <(i + l, 
otherwise. 



The reasoning behind this choice of controls is as follows: to bound the limit 
of the quantity in (2.2), using formula (2.5), by Id{tp*) + h{ip*), we would like 
to specify the controls in form /n)\j /n). Such a choice, as we will see, 

also ensures that the adapted sequence X'^(j) is close to tjj*{j/n). However, 
the adapted process, as it is random, may get too close to a boundary. When 
this happens, not often it turns out, to bound errors, we specify that the 
controls take the cost-free form of the natural evolution sequence. Also, to get 
past this boundary layer initially, ip* has been built as a step function so that 
the adapted process must follow a deterministic trajectory up to time [5n\ . 
Define X"(0) = c'^, and X"(j + 1) = X"(j) + iY"(j) for j > where 

^^(Y"(i) G (iy|X"(0), . . . , X"(i)) = v^{dy;X.^{0), . . . , X"(i)). 

Thus, for j > 0, X"(j) = ^ J^jZo Y"(/) + 0^^. It will be useful later to note the 

total weight Ztoii + < (i/?^ + c) + P{j/n){j/n + c)_and, for 

< j < [Sn\, as mentioned X'"(j) is deterministic and X"(j) = 

hEiZoriiM + c^. 

Define now, for each n > 1, the martingale sequence for < j <n 

1^ 



M-(j) : = 



n 



(Y"(/)-E(Y"(0|X"(/))) 



1 

^"(^■)--E^(Y"(oix"(o) 



1=0 
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Let 



Tn'-=nN min<j \bn \ <l<n: X"(/) < ^(ej(5 + q 



for some <i < d+ l>. 



Then, r„ > \6n] is a stopping time, and the corresponding stopped process 
{M"(j A Tn)} is also a martingale for < j < n. Let now 



A. 



sup |M"(,Ar„,)|>^}. 



0<j<n. 
8 



Lemma 2.8. For n>6 , on the set A^, we have Tn = n. 

Proof. From the definition of {v'j} and r„, we have E'(Y'"(/)|X"(/)) = 
■ip*{l/n) for < / < j A r„ - 1 and j > \6n] . Then, on A^, by (2.14), we have 

Xr(jAr„,)>c. + i'x; ^(Yr(0|X"(0) 
■ =0 



+1 



4ni/8 



jAr„-l 



. f ejjjATn) , 

2 V 



> -(ej(5 + Cj). 



4ni/8 
4nV8 



Hence, Tn = n. □ 



We now observe, by Doob's martingale inequality and bounds, in terms 
of constants C = Cci, that 

P[An]<Cn'/^E\M^{jATnt 



(2.15) 



(Y"(0-^(Y"(0|X"(0)) 



We now state the following almost sure convergence. 
Lemma 2.9. We have 



(2.16) 



lim sup 

ntooo<j<n 



1 



1=0 



a.s. 
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Proof. First, by (2.15) and the Borel-Cantelli lemma, -P(limsupA„) = 
0. On the other hand, on the full measure set Um>i C\k>m.^h since Tn = n 
and £^(Y"(/)|X"(0) = 'tp*{l/n) for < / < n - 1 on by Lemma 2.8, the 
desired convergence holds. □ 



2.2.4. Step 4- We now argue the lower bound through representation 
(2.2). Recall the definition of ^(•) in the beginning of Section 2. The sum in 
(2.2) equals 

" ^ n— 1 



E 



j=0 



[Sn] 



3=0 



[j/n),X"(j)' 



(2.17) 



+ E 



+ E 



^ 71—1 



(j));A„ 



j=\Sn'\ 
n-1 



- E R(-7\\p. 



j=\5n'\ 
--A1 + A2 + A3. 



|'(j/ri),X"{i). 



:A^ 



Step 4.1. We treat the term A2 in (2.17)._Recan a{j/n) = {1 + P{j /n)){j/n) + 
c + c/3{j /n) and the "weight" bound on X"(j) in beginning of Step 3. For 
\Sn\ < j <n — 1, 

= ^(^o(V^*(i/n))|Ip^-(./„)^X"{i)) 

X l(X7(j) > {e/A){eid + Ci) for 0<i<d+l). 
Noting (2.5), this is bounded above, using xlogx < for < x < 1, by 



1 



n 



log p{j n) + (1 -p j/n — 



\j=0 



n 



logi {l-p[j/n)) 



{i + f]{j/n))Xf{j) 
a{j/n) 



r - 



n 
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yd 

X l(X^(i) > {e/4){ei5 + Ci) for 0<i<d+l). 



Eto(^ + /g(i/^))XF(j) 



Given bounds onp, /5 in (ND), as < < 1, we have d < X]j=i[V'*]i < d+ 1 
and 
d 

Y,{^ + /3(i/n))Xr(j) < a{j/n) -{d+l + ^{j /n))%^,^,{j) 

< a{j/n) -{d+l + P{j/n)) ■ {e/A){ed+i8 + Q+i), 
the relative entropy is further bounded by a constant Cd- Thus, for large n, 



(2.18) 



A2<Cd-P 



^supjM'^(,Ar„)|>^^,/, 



<e. 



Step 4.2. Now, for the term Ai in (2.17), we recall for j < I6n\ that 
X"(j) = ^ YliZo ip*{l/n) + c'^ is deterministic. Also note, for < i < c?, that 
^p*{t) = ii on ti < t < tj+i, and i^*{t) = id+i on = t_i <t<to (cf. near 
Lemma 2.5). Thus, for < j < \_5n\ , denoting f_i = f^+i, we may write 

-RKI|P^-(j/n),X"(i)) 



1 



n I n 



1=0 



i-l 



[ti+in] - [tin] j - [tjn] ^ 
H H U + c , tj 



i=-i 



n 



n 



where, for i = —1, the empty sum in the argument for L vanishes. Com- 
paring with the proof of Lemma 2.6, this expression, given bounds on p,/3 
in (ND), is bounded, for < j < [Sn\, except when c'^ = and c / 0, in 
which case a "— log(j'/n)" term appears in the i = —1 term. But, since 

— (1/n) Ylj^=i log(i/^) ^ ~ /o^los(^) dt, its contribution is still small. Hence, 

(2.19) Ai < e{6) where e{6) ^ as 5 ^ 0. 

Step 4.3. We now estimate the last term ^3 in (2.17). For n > 6~^, by 
Lemma 2.8 and definition of L (2.5), 

n-l 



A'^<E 



1 



n 



j=\5n] 



n 



; K n {r„, = n} 
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< E 



lnt\ 



n 



lnt\ 



n 



dt; n B„ 



where ]B„ = {Xf (j) > (6'/4)(ei(5 + q) for < i < d + l,j > \6n]}. On the 
event n B„, ^(Y"(/)jX"(/)) = Tp*{l/n) for / > 0, and so |X"([ntJ) - 
^*{[nt\/n)\l{A^nMn) ^0 for each reahzation from (2.13) and (2.16). 

Also, from the form of L (2.5), as ip* is a step function, (2.8), and bounds 
and piecewise continuity of p,f3 in (ND), we may bound as in Step 4.1 and 
observe 



2Cd > 



lnt\ 



n 



.X"(LntJ),V;* 



[nt\ 



n 



i(A^,nB„) 







for almost all t and each realization. Hence, by bounded convergence theo- 
rem, with respect to dP x l{[6,t])dt, 



(2.20) 



limsup^3< / L{^{t),ij*{t),ij*{t))dt. 



2.2.5. Step 5. Finally, by (2.13) and (2.16), lim„_,oo /i(X"(-)) = h{'4>*{-)) 
a.s. in the sup topology, and by bounded convergence lim„,_^oo-E[/i(X"(-))] = 

Kr{-)). 

We now combine all bounds to conclude the proof of (2.6). By (2.2), 
bounds (2.18), (2.19), (2.20) and nonnegativity of L, we have 



lim sup < lim sup E 



^ n— 1 



<2e 



L{m,r{t),r{t))dt+h{r). 



Then, by Lemma 2.6, we obtain (2.6). 



3. Proof of Theorem 1.4. The proof of Theorem 1.4 follows from the 
following two propositions, and is given below. We first recall the projec- 
tive limit approach, following notation in [20], Section 4.6. Define, for < 
i < j, yj = C([0, l];MJ+2) and Vij-yj ^ by ((/?o, ■ • • , V'j+i) ^ (v^c- ■•, Vi, 
'}yi=i+i'^i) ■ Also define limj^j C nj>o>^« ^ subset of elements x = (x", 

x^, . . .) such that PijX^ = x^, equipped with the product topology. Let also 
Pj : limj^j yj be the canonical projection, pjX = x^ . 
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Since Id are convex, good rate functions on C([0, by the LDPs 
Theorem 1.2 and [20], Theorem 4.6.1, we obtain the following proposi- 
tion. Recall the notation in Theorem 1.2. For n > 1, let ;f"'°° = (X"'°, 
X"'i,...). 



Proposition 3.1. The sequence {X'^'°°] C limy,- satisfies an LDP with 
rate n and convex, good rate function 

{ sup{/rf(prf((^))}, when Lp G limj^j, 

y oo, otherwise. 

To establish Theorem 1.4, it remains to further identify J°°. Recall Td C 
C([0, 1]; M'^"'"^) are those elements 99 = {ipQ, . . . ,ipd,<^d+i) such that: 

93(0) = c"^, each c^j > is Lipschitz with constant 1 such that < [(p{t)\i < 1 

for < i < d, Zt=o ^^ii) = 1> and Zt=o mit) = Eto(l " < 1 

almost all t. 

Let also T* C limy,- be those elements (p = {ip'^,ip^, . . .) such that ip'^ € for 

d>0. Since {Td}d>o are compact sets, it is a straightforward exercise to see 
that F* is compact. Define Ld{pd{^{t))) equal to 



(l-[¥''(t)]o)lo: 



1 - [/(i)]o 



r,(t) + (l- r,(t)) 



2=1 



(i+/3(t))Vf(t) 



l-L PlIJj(i+^(t))t+c+c/3(i) 



. — (it. 

{l+l3{t))t+c+cl3{t) > 



Proposition 3.2. T/ie rate function J°°{^p) diverges when ^T* . How- 
ever, for ip € F*, linid^cx^ Ldipdi^it))) exists for almost all t, and we can 
evaluate 

J°°{^)= [\mLd{pd{^mdt. 

Jo d-tco 

Proof. First, from the definition, J°°{{p) diverges unless 99 G F*. Next, 
for G F* and almost all t, we argue 

(3.1) Lr{pr{^{t))) <Ls{ps{^{t))) when r < s. 
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It will be enough to show from the form of the rates the following: 



i=0 



i=0 



^1 _ ^(f\\(^ _ SU(»+/3W)yn*) - 

PWJIJ^ (l+/3(t))t+c+c/3(t) . 



Consider now /i(x) = xlogx which is convex for x > 0. Under conventions 
(1.3), for nonnegative numbers, Oj and bi, we have 

Yli=p , Yli=p ,( Yli=p , / bi Oj \ 



Yli=p bi Yli=p b'i \ Yli=p bi J y .^^ Yli=p bi bi J 

bi /ai\ J2!=r>ailog{ai/bi) 



i=p 



- $Z V'' b"'\ b- I V b- 



We now finish the proof of (3.1) by applying the last sequence, with p = r + l 
and q = s + 1, to 

(l-[ip'{t)]j, forr + l<i<s, 

s 

l-Y^{l-[^^{t)],), for j = s + l. 



i=0 



and 



b 



Finally, given Ld[pfi{ip{t))) > is increasing in d, the identification of J°° 
in the display of the proposition follows from monotone convergence. □ 

Proof of Theorem 1.4. Let r°° C ni>oC([0, 1];M), endowed with 
the product topology, be those elements ^ = (^O)^i) • • •) such that: 

^j(O) = Cj, ^j(t) > is Lipschitz with constant 1, < [C(i)]i < 1 for i > 0, 

and |E.>oeiW = 1 andlimrf[Eto^e.(t) + (d+l)(l-K(t)]d)] =E.>o(l- 
[i{t)]i) < 1 for almost ah t. 
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We now show that r°° and T* are homeomorphic. Hence, as T* is com- 
pact, r°° would also be compact. (We note, one can see directly that T°° is 
compact.) 

Define the map F : r~ ^ F* by 

F{0 = if, • • • • ■ •> where C'' = (^o, ...,^d,t + c- [^U) G T,. 

In verifying the last inclusion, note Yli=o'^iii^) + {d + 1)(1 — [C(0]rf) = 
Eto(l - < Ei>o(l - < 1- We now argue that F is a bi- 

continuous bijection. 

Indeed, we first note that : T* — )> T°° is given by 

In checking F-'^{ip) G r~, note for 99 G T* that limrf^lrloll " ELoV^K*)) = 
limrf^^^Q(l — [^'^{t)]i) < 1. Then, by bounded convergence with respect to 
the last term in the previous series, ^^T^d{t + Yli=o — Yli=o ^li'^)) = li™(i(i + 
EtoC^-[/Wy =0, and soE i>o¥'K*) — t + c. Finally, it is not difficult 
to see that F and F~^ are both continuous in the product topology. 

Now, G r*, X"'°° G r~, and F(X"'°°) = ;f"'~ for n > 1. Hence, 

through the action of F, the LDP for ;t'"'°° translates to the LDP for X"''"^. 
We now identify the rate function. Given Propositions 3.1 and 3.2, for a 
degree distribution ^ G r°°, we identify its rate as /°°(^) = J°°(F(^)). Since 
r°° is closed, and therefore distributions ^ ^ r°° can never be attained by 
X"'°°, we set = cxD in this case. Last, by properties of F, as J°° is a 

convex, good rate function, one obtains readily is also a convex, good 
rate function. □ 

4. Proof of Corollary 1.7. We verify some properties of in the next 
lemmas and conclude the proof of Corollary 1.7 at the end of the section. 

Lemma 4.1. The ODE (1.5) has a unique Caratheodory solution C,'^ . 

Proof. Any Caratheodory solution to ODE (1.5), given the assumption 
p, /? are piecewise continuous, is piecewise continuously differentiable. Since 
the defining ODEs are linear, one can solve them, and so the solution is 
unique and given by = (Co(i), Ci(*), • • • , C<i+i(*)) where, for t G [0, 1], 

Co(t):=coMo(0,t)+ I {l-p{s))Mo{s,t)ds, 
Jo 

Ci{t) :=ciMi(0,t) 

(4.1) + t(pi^) + (^-Pi^)) n^J^l^t^~\ R( ^ M,{s,t)ds, 

Jo V [1 + (3{s))s + c + c/3{s)J 
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Ci{t) ■.= CiMi{0,t) 

'"^"^^ {l + j3{s))s + c + cl3{s 





for 2 <i <d and 



t-f Jo {l + (3{s))s + c + c(3{s) 



i=0 

Here, for <i <d, 



{l-p{u)) du 
(1 + I3{u))u + c + cl3{u) 



□ 



Mi{s, t) := exp 
Lemma 4.2. We have G T^, and moreover 

oo oo 

J]]Ci(i) = i + c and ^<i(t) = t + c. 

i=0 i=0 

Proof. First, from properties of the ODE system and the piecewise 
continuity assumption on p,P in (ND), Q > 0, Q is Lipschitz with constant 
1 and moreover piecewise continuously differentiable, and < [C(i)]« ^ 1 for 
i > 0, and Yli=o Ci{t) + Cd+i{t) = t + c for d > and almost all t. We postpone 
proving J2i=oi^ — [C(*)]i) ^ 1 for d > and a. a. i, which would complete the 
argument to show ^'^ G F^, until the end. 

We now show X]j>Q Ci(^) = t + c. From the defining ODEs (1.5), for > 1, 

we have 1 - EloUt) = (1 - p{t)) rMMmt) , and hence 



ds. 



(4.2, *+i:c.(o)-f:<.w^/'(i-p(.))-^<"+«»»^-(' 



+ (3{s))s + c + c^{s) 



We obtain, as the integrand on the right-hand side is nonnegative, that 
Ei=o Ciit) < t + E -lo Ci < t + c for all t > and iV > 1 where we recall from 
(LIM) c = ^^QCj. In particular, ^^ix) Jq ds < t. Also, the right-hand 
side of (4.2), after a calculation, is bounded above by ^^^^J -^^ fo^ji^ds. 
Hence, since by nonnegativity and (LIM) the right-side of (4.2) has a limit, 
this limit must vanish and X]i>o = ^ + c 

Next, to establish X]j>o^Ci(^ = t + c, again from the ODEs, for > 1, 

(4.3) 

_ .-I _ . .. (iV + l)(jV + /3(t))C7v(t) 
^ (l + /3(t))i + c + c/3(t) ■ 
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From nonnegativity of d and '^iZo Ci = ^ + c, we bound the right-hand side 

of (4.3) hyp{t) + {l-p{t))^^^§±l^^. Let s^it) := E£o<^W- Then, 

SNit) <p{t) + (1 - p{t))jjf^^^^0^;^^. Since, SAr(t) is piecewise continu- 
ously differentiable, we have, by Lemma 4.3, that S7v(t) <t + c for t > 
and iV > L Hence, ZZo lo '^ds<At since c < Ac for some A> where 

C = Ei>0^Ci<OO. 

Now, integrating both sides of ODE (4.3), we have 



N N 

1=0 1=0 

rt rt 



,4.4, 



JO 







(l + /3(s))s + c + c/3(s) 
From nonnegativity, our estimates and (LIM), the last integral above has a 
limit. This last integral in (4.4) is bounded above by jVm[n{/3o i} io ^i+c'^ 
and hence its limit must vanish. Then, using X^^^g di^) = t + c, we see s{t) = 
^j>QiCj(t) satisfies the ODE in Lemma 4.3, and therefore s{t) = t + c. 
Finally, to finish the postponed verification, noting (4.2), we have 

ji= 

^l + /5(t))t + c + c/3(t) 



i=0 

^ t + c + ^{t){t + c) 

- il + (3{t))t + c + cf3{t) ~ ■ □ 

Lemma 4.3. The ODE 

x + (3{t){t + c) 



m=Git, m) <.nth a(t, .) = m + (i - ^g^t))t+c+ cm 

and initial condition /(O) = c has unique Caratheodory solution t + c for 
t > 0. 

In addition, if u{t) is piecewise continuously differentiable, ti(0) = uq < c, 
and u{t) < G{t,u{t)), then u{t) <t + cfort>0. 

Proof. Since the ODE is linear and, from the piecewise continuity as- 
sumption on p,f3 in (ND), / is piecewise continuously differentiable, we can 
solve uniquely 

(i_p(c,))/3(s)(s + c) 



f{t) = cexp{B{0,t)}+ [ 

Jo 



P[s) + 



;i+/3(s))s + c + c/3(s) 



exp{i?(s, t)}ds, 
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where B{q,r) = /J" dv. Recall the convention • oo = 0, so 

when c = the first term ce^'-'''*^ = vanishes. However, t + c is a solution, 
and therefore f{t) may be identified as desired. 
The second statement is obtained similarly. □ 

Proof of Corollary 1.7. Any root of Id must be a Caratheodory 
solution to ODE (1.5). Hence, by Lemmas 4.1 and 4.2, G is the unique 
minimizer of I^- The LLNs now follow from the LDP upper bound in Theo- 
rem 1.2 and Borel-Cantelli lemma. Statements about "mass" and "weight" 
of are proved in Lemma 4.2. □ 

5. Proof of Corollary 1.9. Since = for i < d, the proof follows 
from the next lemma. Define, for 01,02,03, 04, 05 > 0, the ODEs, 0(01,02,03, 04, 
05): with initial condition (^(0) = c'^ 

^0(t) = l-Ol-(l-O2)^-^, 

[mi = 1 - (1 - 02)^ • ^ for 1 < i < d. 
1 + 04 t + 05 

One can check that xi^) is the solution to 0(01,02,03,04,05) above for 
< 02 < 1, where 

' / \ (l"°2)(^+°3)/(l+04) 

(5.1) Xi{t) = h{t + 05) + y2aiA—^) iorO<i<d. 
Here, the sequence h = 61(01,02,03, 04,05) is defined by 60 = i+(i-027o" /(1+04) ' 

h. — »l + (l-»2)°3 60/(1+04) J f ■ ^ 9 

"1 - l+(l-02)(l+03)/(l+04)' ^ ^ ^' 

, ^, A (l-02)(^- 1 + 03)7(1+04) 
' \iil + (l-02)(^ + 03)/(l + 04) 

^, r(2 + 03 + (1 + 04)7(1-02)) r(i + 03) 

' r(i + 03) r(i + 1 + 03 + (1 + 04)7(1-02)) 

1 



il + (l+04)/{l-02) • 

The sequence aj^^ = 02^^(01,02,03, 04, 05) is given by ao,o = cq — 60O5, and, for 
«>1, 



<iii = - — : — \ ^ Qt-i £ where < £ < i 



and 
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Recall now the assumption in Corollary 1.9: <Pmin ^p(') < Pmax < 1 

and < /?min < < /3max < oo. 

Lemma 5.1. The systems 0(pmin,Pmax, /3mm, /^max, max{c, c}) andO{pma.^ 
Pmin, /3max; /3min, min{c, c}) have respective unique solutions C, and C- Then, 
for <i < d and t G [0, 1], with respect to the zero-cost trajectory C'^{t) in 
Corollary 1.7 with initial condition C'^{0) = , we have 

mi < [Cm. < mi- 

Proof. The proof that ( and ( are the unique solutions uses a similar 
argument to that in the proof of Lemma 4.1. We now establish the inequality 
in the display with respect to C as an analogous proof works for ^. We use 
induction to see that [C]i > [C]i for <i < d. 

Since C(0) = C(0) = C^, from ODEs, (^(pminj^max, /3min, Anax, max{c, c}) 
and (1.5), we have 

(5.2) eo(t)-Co(t)>p(t)-Pmm + (l-p. ^ /?mm(C0(t)-C0(t)) 



(1 + /5max) (i + max{c, c}) ' 

(^ + /3min)(0(t)-0(i)) 



(5.3) [CW]-[CW].>(l-Pmax.(,^^_)(,^_^,^^^). 

For i = 0, suppose Co(0 < Co(0 for some t. Then, by continuity, we may 
assume that Co{t) < Co{t) for all t G (toi^i] for some < to < ti < 1, and 
Co(^o) = Co(^o)- We may further arrange to,ti, from the piecewise continuity 
assumptions in (ND), that p,f3 are continuous on (to,ti)- From the mean 

value theorem, we find a G {to,ti) such that Co(^') < Coif), which contra- 
dicts the ODE (5.2) as it gives Cj){t') - Co{t') > 0. Therefore, Co > Co- 

Now, for 1 <i < d, suppose [C(i)]i < [C(*)I« for some t. By induction hy- 
pothesis ([C(-)]i-i > [C{-)]i-i), we must have Ci(*) < Ci{t)- Since [C(-)]i> [C{-)]i, 
Cj(-) and Cj(0 are continuous functions, as for the case i = 0, we may assume 
[C(t)]j < [C(0]j ^-^d Ci{t) < Ciit), and p,l3 are continuous for all t G {to,ti) for 
some < to < ^1 ^ 1) and also Cj(^o) = Ci(^o)- By the mean value theorem 
for [C{t)]i - [C{t)]i, there is t' G (to,ii) such that [C(tO]i < [Cit%- But (5.3) 
gives [C(t% - [C{t')]i > 0, a contradiction. Therefore [C]i > [C]i. □ 

Proof of Corollary 1.9. Given Lemma 5.1, we need only detail 
the solutions C and C when the initial configuration is "small" and "large," 
respectively. To this end, when the initial configuration is "small" (cj = 0), C, 
C are linear, namely Ci{t) = bit, and Ci(t) = kt, where bi := ?)i(pmin,Pmax, /5min, 
/3max,0) and b. := bi{p 

max 5 Pmin ; /3max,/3mm,0) [cf. (5.1)]. 

On the other hand, when the initial configuration is "large" (cj > for 
some 0<i<d + l), astloo, Ci{t) = (bi + o{l))t and Ci(i) = {bi + o{l))t. □ 
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